# Continuous pre-training

Nous Consilience 40B is a 40-billion-parameter generative text model, pre-trained from scratch in a decentralized manner, supporting multiple languages and representing diverse human creative output.

Llama 3.1 Swallow is a series of large language models built on Meta Llama 3.1. It enhances Japanese language capabilities through continuous pre-training while retaining English language capabilities.

Llama 3.1 Swallow is a series of large language models built on Meta Llama 3.1. It enhances Japanese capabilities through continuous pre-training while retaining English capabilities.

Llama-SEA-LION-v3-8B is a multilingual large language model optimized for Southeast Asian languages, supporting 11 Southeast Asian languages and having undergone continuous pre-training on approximately 200 billion tokens.

This is a Japanese continuous pre-trained model based on Mistral-Nemo-Instruct-2407, focusing on Japanese text generation tasks.

A Japanese-optimized large language model built on Meta Llama 3, enhancing Japanese capabilities through continuous pre-training and improving instruction-following abilities through supervised fine-tuning.